System and method for scheduling  packet transmission on a client device using traffic classes and opportunistic behavior

ABSTRACT

A client device having a networking layer and a network driver layer for transmitting network packets comprising: a plurality of transmit queues configured at the network layer, each of the transmit queues having different packet service classifications associated therewith; a classifier module for classifying packets according to the different packet service classifications, wherein a packet to be transmitted is stored in one of the transmit queues based on the packet service classifications, wherein one or more of the traffic service classifications comprise opportunistic traffic classes and other traffic service classifications comprise non-opportunistic traffic classes; and a socket layer comprising socket transmitting data from an application associated with each traffic service classification, wherein a socket associated with an opportunistic traffic class is automatically suspended in response to detecting a network interface of the client device being throttled.

CLAIM TO PRIORITY

This application is related to, and claims the benefit of U.S.Provisional Patent Application No. 61/595,003, filed Feb. 3, 2012,entitled “SYSTEM AND METHOD FOR INTELLIGENT NETWORK QUEUE MANAGEMENT”,by Cahya Masputra, et al., which is hereby incorporated by referenceherein in its entirety.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to concurrently filed U.S. Application Nos.______ filed Sep. 15, 2012 by Cahya Masputra, et al., assigned to Apple.

An embodiment of the invention is directed to managing data networkcommunications in a client device. Other embodiments are also described.

BACKGROUND

A data network allows people to communicate with each other and obtaininformation from various sources on the network, using their respectiveclient devices that are “on” the network. For example, a Web browserapplication program, running in a user's workstation or laptop computer,can connect with a Web server to download a Web page. The connection mayspan several intermediate nodes or hops of the network, which mayinclude specialized computers such as routers. These devices candiscover routes between the end nodes through which they can forwardmessages that have been broken up into packets of data. Each node may beassigned a unique or global address, such as an Internet Protocol (IP)address. The Internet is a well known global inter-network in whichnetworks of computers are connected to each other via routers.

Computer network protocols have a layered architecture. Typically, theupper most layer includes the functionality provided by an applicationprogram, such as a Web browser. This is the layer that, at least in theend nodes, may initiate a connection between two computers over anetwork. Thus, for example, a user may select a desired Website on hiscomputer. The Web browser (running in that computer) starts a procedurethat results in a connection being made with a server that is associatedwith the selected Website. The Web browser sends the request “down”through a series of functions referred to as an Internet protocol suiteor Transport Control Protocol/Internet protocol (TCP/IP) stack. Thisstack of protocols is typically implemented in software at its higherlayers, often as part of an operating system (OS) program running in theclient device. Once the selected Website has been translated into an IPaddress of a Web server, the server is contacted over the Internet, andan appropriate connection is made with an upper layer program of asimilar protocol suite implemented in the Web server.

To use the connection, the TCP/IP stack in the user's computerencapsulates a request message from the Web browser, in this example, arequest identifying the Web page. The message may be encapsulated morethan once, by several vertical layers on its way down in the protocolstack, including a network access layer. It finally arrives at thelowest layer of the client device, namely the physical layer (which istypically deemed to be a part of the network access layer).

After leaving the physical layer of the user's computer and then makingits way through one or more hops in the network, the message from theWeb browser arrives in the Web server, and is passed “up” the protocolstack in the Web server to a program that is deemed a peer of the Webbrowser. The peer program may then respond to the message, by causingthe data for the requested Web page to be collected and sent back to theuser's computer through the existing network connection. The data isbroken up into multiple messages or packets, and is sent in a manneranalogous to how the request message was sent.

An application program can have several applications or processes thatare executed by one or more processors in the user's client computer.Each individual application may generate different types of network datatraffic which may have different packet loss, latency, and flowelasticity requirements. By way of example, a social networkingapplication may communicate control data, text, audio, and video overthe network, each of which have different requirements with respect tothe above variables. Each application is typically provided with its ownport or group of ports to communicate this data, though they may allshare the same lower layer network resources in the user's computer. Incurrent implementations, the routers interconnecting each client deviceto a particular destination node (i.e., another client or a server) overthe network include large transmit and receive buffers. As such, thereis little or no packet loss and client devices are typically permittedto transmit packets without regard to flow control, resulting in “bufferbloat” within the router queues. Protocols such as TCP are self-tuningprotocols which determine congestion and modify transmission speed basedon detected packet loss. When packet loss is mitigated using largebuffers, the TCP protocol

Additionally, in current client-side implementations, the buffering ofpackets occurs at the driver level. The TCP/IP stack simply pushespackets down to the driver and the driver manages its own transmit andreceive queues. Because of the large amount of buffering performed atthe driver level within the client (an Ethernet driver can buffer up to4000 packets in a queue prior to transmission), the networking stack isnot provided with accurate network/congestion information. As such, whatis needed is a more intelligent mechanism for performing network queuingwithin a client device in which a feedback channel is utilized usedbetween the driver layer and the network stack layer.

SUMMARY

A client device having a networking layer and a network driver layer fortransmitting network packets comprising: a plurality of transmit queuesconfigured at the network layer, each of the transmit queues havingdifferent packet service classifications associated therewith; aclassifier module for classifying packets according to the differentpacket service classifications, wherein a packet to be transmitted isstored in one of the transmit queues based on the packet serviceclassifications, wherein one or more of the traffic serviceclassifications comprise opportunistic traffic classes and other trafficservice classifications comprise non-opportunistic traffic classes; anda socket layer comprising socket transmitting data from an applicationassociated with each traffic service classification, wherein a socketassociated with an opportunistic traffic class is automaticallysuspended in response to detecting a network interface of the clientdevice being throttled.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments of the invention are illustrated by way of example andnot by way of limitation in the figures of the accompanying drawings inwhich like references indicate similar elements. It should be noted thatreferences to “an” or “one” embodiment of the invention in thisdisclosure are not necessarily to the same embodiment, and they mean atleast one.

FIGS. 1A-B illustrate block diagrams of a client device with anetworking stack in accordance with embodiments of the invention.

FIG. 2A-B illustrates two methods in accordance with embodiments of theinvention.

FIGS. 3A-B illustrate communication between a networking stack and adriver for transmitting data packets in accordance with two differentembodiments of the invention.

FIG. 3C illustrates a client-side software architecture according to oneembodiment of the invention.

FIG. 4 illustrates an exemplary sender thread and an exemplary starterthread.

FIG. 5 illustrates an exemplary class queue instance and scheduler.

FIG. 6 illustrates one embodiment in which service classes are mapped toqueue instances.

FIG. 7 illustrates another embodiment in which service classes aremapped to queue instances.

FIG. 8 illustrates an embodiment in which socket traffic classes aremapped to service classes.

FIG. 9A illustrates a worst-case scenario of link-share distribution inone embodiment with the built-in QFQ scheduler configuration.

FIG. 9B illustrates an example of a packet filter (PF) configuration inaccordance with one embodiment.

FIG. 10A illustrates flow control using a flow hash in accordance withone embodiment.

FIG. 10B illustrates a current set of bins and a shadow set of bins usedby the queuing algorithm in one embodiment.

FIGS. 11A-B illustrate communication between a networking stack and adriver for receiving data packets in accordance with two differentembodiments of the invention.

FIG. 12 illustrates an exemplary work loop thread, poller thread, andDLIL input thread.

FIG. 13 illustrates an exemplary set of thread data in accordance withone embodiment.

FIG. 14 illustrates an application programming interface (API) employedin one embodiment.

FIG. 15 illustrates a plurality of services with APIs employed in oneembodiment.

FIG. 16 illustrates one embodiment of a client-side data processingdevice.

FIG. 17 illustrates another embodiment of a client-side data processingdevice.

DETAILED DESCRIPTION

An embodiment of the invention is directed to a computer-implementedmethod for active queue management for networking applications executedon a client device.

FIG. 1A is a block diagram of a client computing device 101 on whichembodiments of the invention may be implemented. The illustrated clientdevice 101 executes a plurality of applications 105-107 whichcommunicate with servers 120-121 and other client devices 122 over anetwork 150 such as the Internet. The servers 120-121 may include (byway of example and not limitation) Web servers, email servers, instantmessaging servers, and file servers. In one embodiment, the applications105-107 make calls to a networking application programming interface(API) 208 exposed by the networking stack 102 to access networkingresources provided by the networking stack 102.

The networking stack 102 of this embodiment includes queue managementlogic 115 for managing a plurality of networking queues 110-112 onbehalf of each of the applications 105-107. A packet scheduler 116 inthe networking stack schedules packets to be transmitted and receivedfrom/to each of the queues based on packet classifications (as describedin greater detail below). Although illustrated as separate modules inFIG. 1A, it will be appreciated that the queue management logic 115 andscheduler 116 may be implemented as a single integrated software module.

In one embodiment, each of the queues 110-112 managed by the queuemanagement logic 110-112 includes a send queue for storing outgoingnetwork packets (e.g., TCP/IP packets) and a receive queue for storingincoming network packets. The send and receive queues are manageddifferently for each of the applications 105-107 based on the networkingrequirements of each of the applications 105-107. For example, differentapplications may have different packet loss, latency, and flowelasticity requirements, all of which are monitored and managed by thequeue management logic 115. In one embodiment, the networkingrequirements of each application 105-107 are specified ahead of time(e.g., when the application initially registers via the API 108) and thenetworking packets for that application are managed by the queuemanagement logic 115 based on the specified requirements. By way ofexample, a Web browsing application is typically more latency-tolerantthan a real-time video chat application. Consequently, the Web browsingapplication will be associated with a different queue having differentspecified service level than the real-time video chat application.

In one embodiment, each driver 150 installed on the client deviceincludes queue management logic 155 for managing a set of driver queues151-153, each of which may also be associated with a different servicelevel. In addition, each driver may have its own scheduler 160 forperforming packet scheduling at the driver level and its own queuemanagement logic 155 for managing the queues 151-153. As with thenetworking stack 102, the driver queue management logic 155 and driverscheduler 160 may be implemented as a single logical module (rather thanseparate modules as illustrated in FIG. 1). In one embodiment, eachdriver 150 may either choose to manage packet scheduling on its own(referred to herein as “driver managed” scheduling) or may rely on thepacket scheduler 116 and queue management logic 115 of the networkingstack 102 for packet scheduling/queuing.

By way of example, and not limitation, Ethernet drivers and cellularwireless (e.g., 3G, 4G) drivers may rely on the packet scheduling andqueuing provided by the scheduler 116 of the networking stack 102 while802.11n (Wi-Fi) drivers may use the driver scheduler 160 to managepacket scheduling and queuing. In one embodiment, the Wi-Fi driverimplements Wireless Multimedia Extensions (WME) (also known as Wi-FiMultimedia (WMM) standard), to schedule network traffic according to thefour priority levels of voice, video, best effort, and background.However, the underlying principles of the invention are not limited toany particular networking standard.

In one embodiment, the Wi-Fi driver is capable of dynamically switchingbetween driver-managed scheduling and scheduling at the network layer.For example, when on an 802.11n network which supports WME, the drivermay choose driver-level scheduling but when on an 802.11b or 802.11gnetwork, the driver may choose scheduling at the network layer. In oneembodiment, when utilizing network stack-managed scheduling, the networkstack 102 will notify the driver when a packet is ready to be dequeued.The driver will then dequeue and transmit the packet (as described ingreater detail below).

As illustrated in FIG. 1A, in contrast to prior implementations in whichpackets are pushed from the networking stack to the driver and bufferedin the driver regardless of network conditions, in one embodiment of theinvention, continuous feedback 171 is provided between the driver layer150 and the networking stack 102. The feedback from the driver 150 tothe networking stack ensures that the networking stack 102 is aware ofnetworking conditions of the communication link managed by the driver150 and may perform packet scheduling/queuing based on this knowledge.In one embodiment, the networking stack 102 implements packet schedulingand queuing intelligently as described herein, based on the detectednetworking conditions. Similarly, the feedback signal from thenetworking stack 102 to the driver 150 notifies the driver of theconditions within the transmit/receive queues 112 of the networkingstack (e.g., such as when a new packet is ready to be transmitted from aparticular queue).

As illustrated in FIG. 1B, scheduling may be employed for at the networkstack layer for some communication channels and at the driver layer forothers (e.g., some WiFi channels as discussed above). In particular, inthe illustrated embodiment, packet scheduling may be performed in thenetwork stack layer 102 for the communication channels to servers 120and 123, and client 121, whereas driver-managed scheduling may beperformed for the communication channel to server 122. In addition, asshown in FIG. 1B, a single application 105 may have different types ofdata traffic assigned to different queues which support different packetloss, latency, and flow elasticity requirements. For example, aparticular application may open a TCP/UDP socket to communicate controldata, text, audio, and video, each of which have different requirementswith respect to the above variables. As such, one type of data (e.g.,control data) may be queued in queue 113 associated with a first serviceclass and a second type of data (e.g., interactive video) may be queuedin queue 110 associated with a second service class. Additionally,different applications may queue data in the same queue associated withthe same service class. For example, applications 105 and 106 may queuecontrol data in queue 110 associated with a service class for controldata and applications 106 and 107 may queue interactive video data inqueue 111 associated with a service class for interactive video data.

Additionally, It will be understood that, depending on networkconnectivity (e.g., whether the client 101 is coupled to Ethernet orWifi) and other network variables, the client device 101 may utilizeonly network-layer queue management and/or scheduling or onlydriver-managed queue management and/or scheduling while still complyingwith the underlying principles of the invention.

A method in accordance with one embodiment of the invention isillustrated in FIG. 2A. At 201, a packet to be transmitted is receivedat the networking layer of a protocol stack on a client device. If thepacket is associated with a network link that uses driver-managedscheduling, determined at 202, then at 203 the packet is provided to thedriver layer. The driver layer then queues, schedules and transmits thepacket at 204. If, however, the packet is associated with a network linkthat performs packet scheduling at the network layer then, at 205, thenetwork stack queues and schedules the packet for transmission. At 206,the driver is notified when the packet is ready to be transmitted and,at 207, the driver transmits the packet.

The client computing device 101 may be a desktop computer, a notebook orlaptop computer, a video game machine, or other consumer electronicdevice. In some embodiments described herein, the client device 101 is aportable wireless device that may include two-way voice and video, emailmessaging, and media playback functions. The communication path betweenthe client device 101 and a server, in this example, has a wirelesssegment between the client device 101 and a wireless base station (e.g.,a cell tower or Wifi access point). In the Internet reference model, thenetworking stack 102 client device 101 communicates with a networkaccess gateway via the base station, in accordance with any suitablewireless communications network access protocol, some examples of whichare given below. The other client device 122 may be reached via thecombination of another base station and gateway. On top of the networkaccess layer are the internetworking layer (e.g., defining an InternetProtocol, IP, address for each node on the network), the transport layer(e.g., Transport Control Protocol, TCP, performing host-to-host flowcontrol and the opening and closing of connections), and the applicationlayer (e.g., application programs and process protocols such as HTTP,SMTP, and SSH).

FIG. 2B illustrates an embodiment of the invention in which a packet istransmitted using either driver-managed scheduling or networkstack-managed scheduling. At 251 a packet to be transmitted is generatedby a particular application over an open socket connection. For example,an interactive video application may generate a video packet to betransmitted to another client. At 252, the network layer queues thepacket in a specified service class based on the type of packet. Forexample, as discussed below, 10 different service classes may be definedto queue data for 10 different types of data traffic. Thus, if thepacket is an interactive video packet, it may be queued in a serviceclass queue for interactive video. Similarly, if the packet containscontrol data, it may be queued in a service class queue for networkcontrol.

Regardless of how the packet is queued, it may be dequeued differentlydepending on whether the driver- or network stack-managed scheduling.For driver-managed scheduling, determined at 254, the driver performs adequeue operation from a specified service class at 255. For example, ifthe driver is implementing 802.11n, then it may choose to perform thescheduling using the four service classes defined by WMM (see, e.g.,FIG. 7 illustrating the 10:4 mapping between service classes and queueinstances). Alternatively, for other network interface types (e.g.,Ethernet, 3G, etc) scheduling may be performed at the network layer(see, e.g., FIG. 6 illustrating a 1:1 mapping between service classesand queue instances). Thus, at 260, the network layer performs thedequeue operation from the selected service class. At 270, the packet isprovided to the driver layer which transmits the packet at 271.

Thus, it can be seen from the above that, in one embodiment, when apacket needs to be transmitted, the packet is passed into anetwork-layer scheduler configured for the network interface. Thescheduler extracts the service class from packet; service classdetermines the queue instance to enqueue the packet on. The packet thengets enqueued onto the corresponding queue instance; the packet may bedropped if queue is full or flow controlling (the decision todrop/enqueue is left to the queuing discipline/algorithm (e.g. SFB) asdescribed below). The driver is notified that there is work to do. Atsome point, the driver dequeues a packet. The queue instance needs to beidentified. If the interface is configured for “network stackscheduling”, the scheduler selects eligible queue to be serviced. If theinterface is configured for “driver scheduling”, the driver indicates tothe scheduler the queue to be chosen for service. Once a queue instanceis identified, a packet, if available, is dequeued from the queue. Thepacket is then handed to the driver for transmission over medium whichtransmits the packet.

As indicated in FIGS. 1A-B, in one embodiment, continuous feedback 172is provided from the networking stack 102 to each of the applications105-107 (as indicated by the dotted arrows) and is used to provide flowcontrol for the network flow to/from each of the applications 105-107.For example, when the transmit queue 110-112 for a particular TCP or UDPsocket has reached a specified threshold, a feedback signal is generatedto instruct the respective application 105-107 to suspend or reduce newpacket transmissions.

FIGS. 3A-B illustrate different network driver models in accordance withdifferent embodiments of the invention. In FIG. 3A, the application 301sends packets to be transmitted to the network stack 302 (1) which thensends the network packets to the IO networking interface 303 of thedriver (2). In one embodiment, the IO networking interface 303classifies the packet and places the classified packet in an appropriateIO Output Queue 304 based on the packet classification (3). As mentionedabove, for WMM, the classifications may include voice, video, besteffort, and background. The driver 305 then uses its own packetscheduler to dequeue the packet from the appropriate IP output queue 304(4, 5).

In the network stack-managed model illustrated in FIG. 3B, theapplication 301 sends packets to be transmitted to the network stack 302(1) which then classifies the packet (using the classification schemesdescribed below), and places the classified packet in an appropriatesend queue 306 (2). In one embodiment, there are as many different sendqueues 306 as there are packet classifications (e.g., 10 different sendqueues for 10 different packet classifications). The networking stack302 notifies the driver layer (3) when a new packet is ready fortransmission in one of the queues. The IO networking interface of thedriver 307 then dequeues the packet and passes the dequeued packet tothe driver 305 for transmission (5, 6).

FIG. 3C illustrates additional architectural details of the networkinglayer 102 including a packet classifier 202 for classifying packets, anAPI 203 for interfacing with applications 201, an Internet Packet layer206, transport layer 205 (e.g., TCP, UDP), and socket layer 204, apacket scheduler 209 for scheduling packet transmission, a plurality ofclass queues 210, a flow advisory module 207, and a kernel programminginterface (KPI) 211. Each of these components is described in greaterdetail below.

A. Kernel Programming Interface

In one embodiment, the following set of private KPIs are employed:

ifnet_allocate_extended( )

Allocate an ifnet instance which supports the new output model. This isan extended (private) version of the public ifnet_allocate( )KPI, whichrequires the newly-defined ifnet_init_eparams structure to be filled bythe caller. This structure is analogous to the ifnet_init_params, withseveral new fields that are related to the new output model:

ifnet_init_eparams fields Descriptions pre_enqueue( ) If defined, thiscallback is invoked for each outgoing packet for the interface. Thedriver may perform last-minute changes on the (fully-formed) packet, butit is responsible for calling ifnet_enqueue( ) to enqueue the packetupon completion. (Some drivers will not need to register this callback,but it is provided just in case.) start( ) This callback is used toindicate to the driver that one or more packets may be dequeued bycalling ifnet_dequeue( ) or ifnet_dequeue_multi( ). This routine getsinvoked when ifnet_start( ) is called; it will be executed within thecontext of a dedicated kernel thread, hence it is guaranteed to besingle threaded. The driver must employ additional serializations ifthis callback routine is to be called directly from another context, inorder to prevent race condition related issues. output_sched_model Thisinforms the networking stack about the packet scheduling requirements ofthe driver. By default, the networking stack will choose a scheduler andthe scheduling policy for the interface. That means when the driverdequeues a packet, the dequeuing decision is controlled by thescheduler; this is the “normal” scheduling model. Alternatively, thedriver may choose to perform its own scheduling. In this “drivermanaged” scheduling model, the network stack simply provides the queueinstances, and the driver is then required to specify the queue fromwhich the packet is to be dequeued from. A use case example of thelatter model is an 802.11 driver which incorporates Wi-Fi Multimedia(WMM); the hardware in conjunction with the access point perform thepacket scheduling. sndq_maxlen The maximum size of the output queue;this is more or less a hint from the driver about the size of itstransmit queue. When set to 0, the network stack will attempt to pick areasonable size (currently set to 128 packets.) output_ctl( ) For futureuse, to provide for a way for the networking stack to inform theunderlying driver/family about changes on the output parameters (e.g.hardware offload, transmit ring size, etc.) {input_bw, The effective andtheoretical input/downlink link rates; input_bw_max} for informationalpurposes. {output_bw, The effective and theoretical output/uplink linkrates. output_bw_max}

ifnet_enqueue( )

Enqueue a packet to the output queue of an interface which implementsthe new driver output model. This is provided for a driver/family whichimplements a pre_enqueue( ) callback.

{ifnet_dequeue, ifnet_dequeue_multi}( )

Dequeue one or more packets from the output queue of an interface whichimplements the new driver output model, and that the scheduling model isset to “normal.”

   {ifnet_dequeue_service_class, ifnet_dequeue_service_ class_multi}( )

Dequeue one or more packets from the output queue of an interface whichimplements the new driver output model, and that the scheduling model isset to “driver managed.”

  ifnet_set_output_sched_model( ) Set the scheduling model to “normal”or “driver managed.” {ifnet_set_sndq_maxlen, ifnet_get_sndq_maxlen}( )Set and get the maximum length of the output queue. ifnet_get_sndq_len() Get the current length of the output queue. ifnet_start( )

Trigger the transmission at the driver layer on an interface whichimplements the new driver output model. This may result in the driver'sstart( ) callback to be invoked, if not already.

{ifnet_set_bandwidths, ifnet_bandwidths}( )

Set and get the uplink and downlink link rates of the interface. Therates may be set by the driver at anytime after the ifnet is attached,whenever the information is available at its layer.

{ifnet_transmit_burst_start, ifnet_transmit_burst_end}( )

Alternative mechanisms to estimate the uplink link rate, when the driveris not able to easily retrieve such information from the hardware. Theseinform the networking stack about the beginning and end of transmissionof a burst.

In one embodiment, a driver that has registered itself as supporting thenew output model (i.e., the network stack managed scheduling) is flaggedwith the IFEF_TXSTART flag.

B. Data Link Interface Layer (DLIL) 208

Starter Thread 402 (one embodiment illustrated in FIG. 4)

An interface which supports the new output model (i.e., network layerscheduling) uses a dedicated kernel thread, the “starter thread” 402,whose job is to invoke the driver's start( ) callback. In oneembodiment, this thread is signaled to run, if not already, wheneverifnet_start( ) is called, as part of enqueuing a packet viaifnet_enqueue( ) allowing the application thread to return immediatelyupon enqueuing the packet to the output queue. It provides a form ofserialization for the driver's start( ) callback, so that the dequeuecan happen in-order. It also reduces complexity at the driver layer, asthe driver may perform certain operations (hardware-related or not) thatmay momentarily block the execution of the thread without worrying toomuch about the impact, as no lock is held by the networking stack whenit executes the driver's start( ) callback from the context of thisstarter thread.

Token Bucket Regulator

Additionally, the network layer managed output model allows for a formof uplink rate limiting at the ifnet layer, when a Token BucketRegulator (TBR) is configured for the interface. By default, aninterface does not have a TBR configured; enabling TBR requires manualconfiguration via ifconfig(8) or pfctl(8). When TBR is enabled, thestarter thread will periodically wakeup (every 10 ms) whenever theoutput queue is non-empty as illustrated at 402 in FIG. 4. During eachperiod, the driver is allowed to dequeue as many bytes as there areavailable tokens; the tokens get refilled at the beginning of eachperiod. The number of tokens are computed according to the rate forwhich the TBR is configured for. One particular TBR implementation doesnot require callouts to be allocated (unlike the approach taken by BSD);because of this, it can accommodate extremely high rates (tens of Gbps)with very low CPU overhead, as the interval is fixed and thusindependent of the callout resolution (10 ms is achievable acrossdifferent platforms.)

Transmit Queues (One Embodiment Illustrated in FIG. 5)

The if_snd member of ifnet holds the transmit queues for the interface.This data structure contains information about the built-in scheduler(type, instance, callbacks), TBR, and optionally, an alternativescheduler.

By default, in one embodiment, the system creates a built-in instance ofpacket scheduler (ifcq instance). As mentioned, the choice of a packetscheduler and its parameters depends on the type of the networkinterface and in some instance the topology of the network as well. Inone embodiment, when a packet scheduler is attached, it stores itsinstance in ifcq_disc, and configures the enqueue( ) dequeue( ) andrequest( ) callbacks to the scheduler's corresponding routines. Forinterfaces which requires a “driver managed” model, a special scheduleris attached, which provides an alternative dequeue_sc( ) instead of adequeue( ) callback. Certain embodiments of these callbacks are asfollows:

Scheduler functions Descriptions ifclassq_enq_func( ) Enqueues a singlepacket to the scheduler. The possible return types are: CLASSQEQ_SUCCESSwhen the packet is successfully enqueued; CLASSQEQ_SUCCESS_FC when thepacket is successfully enqueued on a queue that is asserting flowcontrol; CLASSQEQ_DROPPED when the packet is dropped;CLASSQEQ_DROPPED_FC when the packet is dropped because the queue exceedsits flow control limit; and CLASSQEQ_DROPPED_SP when the packet isdropped because the queue is suspended. ifclassq_deq_func( ) Dequeues asingle packet from the scheduler; this is whatever packet the schedulerdetermines to be most eligible for transmission. The dequeue opcode caneither be CLASSQDQ_REMOVE for normal dequeue, or CLASSQDQ_POLL forretrieving the eligible packet without actually dequeuing it.ifclassq_deq_sc_func( ) Similar to to the regular dequeue callback,except that the caller determines the service class of the packet thatis eligible for transmission. ifclassq_req_func( ) A request callback tothe scheduler. The last parameter depends on the type of request made.The types are: CLASSQRQ_PURGE for purging all queues in the scheduler;CLASSQRQ_PURGE_SC for either purging a specific queue, or optionally aflow from within that queue; CLASSQRQ_EVENT for propagating interfaceevents (changes in speed, MTU, link state); and CLASSQRQ_THROTTLE forapplying throttling parameters for the queue.

One embodiment of the scheduler 116 instantiates N number of classes;each class correlates to a service class and manages a queue instance110-112. Packets are enqueued in one of these queue instances, dependingon how they are classified. When PF_ALTQ support is configured, thebuilt-in scheduler and its parameters may be overridden via the PacketFilter (PF) infrastructure (e.g. by way of pfctl(8)). This provides fora convenient way for different characteristics of the packet schedulingto be modeled (e.g. trying out different schedulers and/or parameters).

Packet Scheduler 116

One embodiment of the packet scheduler module 116 provides entry pointsfor enqueuing and dequeuing packets, to and from one of its class queueinstances 210. In one embodiment, each class corresponds to a queue. Itmanages all of its queues depending upon the scheduling algorithm andparameters.

In one embodiment, a scheduler 116 gets configured and attached to anifnet through one of the following techniques:

1. Built-in (Static): When an interface is attached to the networkingstack, a scheduler is chosen based upon the queue scheduling model thatis requested by the driver. For a “normal” model, the stack creates ascheduler with 10 classes (hence queues). For a “driver managed” model,an instance of a special scheduler with 4 classes is created instead.

2. PF (Dynamic): In one embodiment, the built-in (static) configurationmay be overridden by configuring the scheduler and its parameters viathe PF framework. This requires the PF_ALTQ configuration option to beenabled, and “altq=1” boot-args NVRAM option to be present. In oneembodiment, it is not enabled/allowed by default. However, when enabled,it allows for a convenient and expedient mechanism for experimentingwith different schedulers and parameters.

In one embodiment, the following schedulers are not used by default andare available only in PF:

Scheduling Algorithms Descriptions PRIQ Priority Queuing. This is a non-hierarchical scheduler. It does not require any knowledge of the uplinkbandwidth. This scheduler came as part of the original ALTQ code. FAIRQFair Queuing. This is a non- hierarchical scheduler and is an extensionof PRIQ with added fairness. It requires knowing the uplink bandwidth,and each priority queue must be assigned with a bandwidth. CBQClass-Based Queuing. This is a hierarchical scheduler. It requiresknowing the uplink bandwidth. HFSC Hierarchical Fair Service Curve. Thisis a hierarchical scheduler. It requires knowing the uplink bandwidth,and it is also rather complex though unique, as it simultaneouslysupports real- time, link-sharing and rate-limiting services. It allowsfor each queue to be configured with guaranteed delay characteristics aspart of the real-time service. This scheduler came as part of theoriginal ALTQ code.

In one embodiment, the following schedulers are used by default (i.e.,at the networking stack layer 102):

Scheduling Algorithms Descriptions QFQ Quick Fair Queuing. This is anon- hierarchical scheduler. It does not require any knowledge of theuplink bandwidth, and is extremely simple to configure. It is also themost efficient scheduler amongst those which provide fairness andbandwidth distribution guarantees. This scheduler can also be configuredvia packet filter (PF). In one embodiment, QFQ is used for network layermanaged scheduling. One embodiment of the QFQ algorithm is described inthe document entitled QFQ: Efficient Packet Scheduling With TightBandwidth Distribution Guarantees, attached as Appendix A to this patentapplication. TCQ Traffic Class Queuing. This is a non- hierarchical,passive scheduler. It does not employ any scheduling algorithm, andsimply provides mapping between service classes and groups. Thisscheduler relies on the driver or hardware to perform the actual packetscheduling (i.e., driver managed scheduling), as the driver isresponsible for choosing the service class that is eligible fortransmission.

Mapping

1. 1:1 Mapping

As illustrated in FIG. 6, the QFQ configuration used in one embodimentfor network layer-managed scheduling provides a 1:1 mapping betweenpacket service classes 601-610 and packet queue instances 611-621,respectively. As illustrated, the 10 service levels are roughly dividedinto 4 groups 630, 640, 650, 660, and prioritization is provided withineach group. The groups are defined based upon the characteristics of theclassified traffics, in terms of the delay tolerance (low-high), losstolerance (low-high), elastic vs. inelastic flow, as well as otherfactors such as packet size and rate. As described herein, an “elastic”flow is one which requires a relatively fixed bandwidth whereas an“inelastic” flow is one for which a non-fixed bandwith is acceptable.The illustrated 1:1 mapping allows for the networking stack to achievefull control over the behavior and differentiation of each serviceclass: a packet is enqueued directly into one of the queues according tohow it was classified; during dequeue, the scheduler determines thepacket that is to be transmitted from the most eligible queue.

2. 10:4 mapping

As illustrated in FIG. 7, the TCQ configuration used for driver-managedscheduling in one embodiment provides a 10:4 mapping between 10 serviceclasses 601-610 and 4 queue instances 701-704. This scheduler ispassive, in the sense that it does not perform any kind of packetscheduling, but instead simply provides the queue instances and maps theservice classes to the queues. Because the networking stack of thisembodiment has no control over the scheduling, the queues cannot bedefined with similar characteristics as those for the 1:1 mapping.Instead, the could be perceived as having priorities ranging from low(L) to highest (H+). A packet is enqueued into directly into one of thequeues which represents the service class that it was originallyclassified with. During dequeue, the driver is responsible for choosingthe most eligible service class for transmission. The number of queuesis set to 4 and is an implementation artifact; as mentioned this alsohappens to be the number of 802.11 WMM access categories.

Queuing Disciplines

In one embodiment, a queuing discipline or algorithm module manages asingle instance of a class queue; a queue simply consists of one or morepackets (mbufs). The algorithm is responsible for determining whether ornot a packet should be enqueued or dropped.

In one embodiment, a queuing algorithm gets configured and attached toan instance of a scheduler class through one of the following ways:

1. Built-in (network stack-managed): When a scheduler class getsinstantiated as part of configuring a packet scheduler on an ifnet, aqueuing algorithm is chosen. All classes of the scheduler share the samequeue algorithm (each with its own unique instance.)2. PF (Dynamic, or driver-managed): Alternatively, the built-in (static)configuration may be overridden by configuring the scheduler and itsparameters—including the queuing algorithms for the classes via thepacket filter (PF) framework.

Algorithms

In one embodiment, the following queuing algorithms are not used bydefault, and are available only via PF:

Queuing Algorithms Descriptions DROP-TAIL Basic FIFO with tail-drop. REDRandom Early Drop. This came from ALTQ. RIO RED with IN/OUT bit. Thiscame from ALTQ. BLUE BLUE maintains a drop/mark probability. When aqueue exceeds its allocation, the probability is increased. When a queueis empty, the probability is decreased. Packets are dropped/marked whenthe probability reaches a certain limit. Unlike RED, this algorithmrequires little or no tuning; however, it is not capable of trackingmultiple flows. SFB Stochastic Fair BLUE, essentially a fair variant ofBLUE which hashes flows and maintains different mark/drop probabilityfor each flow, based on whether a flow exceeds its allocation or isidle. The SFB algorithm employed in one embodiment is described in thedocument attached as Appendix B to this patent application entitledStochastic Fair Blue: A Queue Management Algorithm for EnforcingFairness. SFB also provides an additional rate-limiting functionalityfor inelastic flows. The ability of SFB to track multiple flows isimportant for the Flow Advisory mechanism (as well as those derived fromit.) This is an original implementation/ interpretation of the SFBalgorithm designed by the assignee of the present patent application.

Packet Classification

As mentioned, in one embodiment, each outbound packet is enqueued in aclass queue instance that corresponds to the packet's service class. Theservice class assignment, or packet classification, occurs in severalplaces throughout the networking stack. In general, packetclassification can be explicit (opted-in by the application) or implicit(set or overridden by the system).

Explicit Classification:

In one embodiment, an application may classify its traffics by issuingthe SO_TRAFFIC_CLASS option—either sticky via setsockopt(2) or on aper-message basis with sendmsg(2)—using one of the following trafficservice class values, which are illustrated mapped to service classes inFIG. 8:

Traffic Service Classes Descriptions SO_TC_BK_SYS “BackgroundSystem-Initiated”, high delay tolerant, high loss tolerant, elasticflow, variable size & long-lived. E.g., system initiated iCloud ™synching or Time Capsule backup, for which there is no progressfeedbacks. SO_TC_BK “Background”, user-initiated, high delay tolerant,high loss tolerant, elastic flow, variable size. E.g., user-initiatediCloud synching or Time Capsule backup; or traffics of backgroundapplications, for which there is some progress feedback. SO_TC_BE “BestEffort”, unclassified/standard. In one embodiment, this is the defaultservice class. SO_TC_RD “Responsive Data”, a notch higher than “BestEffort”, medium delay tolerant, elastic & inelastic flow, bursty, long-lived. E.g., email, instant messaging, for which there is a sense ofinteractivity and urgency (user waiting for output). SO_TC_OAM“Operations, Administration, and Management”, medium delay tolerant,low-medium loss tolerant, elastic & inelastic flows, variable size.E.g., Virtual Private Network (VPN) tunnels. SO_TC_AV “MultimediaAudio/Video Streaming”, medium delay tolerant, low-medium loss tolerant,elastic flow, constant packet interval, variable rate & size. E.g.,AirPlay ™ playback (both video and audio). SO_TC_RV “ResponsiveMultimedia Audio/Video”, low delay tolerant, low-medium loss tolerant,elastic flow, variable packet interval, rate and size. E.g., AirPlaymirroring, screen sharing. SO_TC_VI “Interactive Video”, low delaytolerant, low-medium loss tolerant, elastic flow, constant packetinterval, variable rate and size. E.g., FaceTime ™ video. SO_TC_VO“Interactive Voice”, low delay tolerant, low loss tolerant, inelasticflow, constant packet rate, somewhat fixed size. E.g., VoIP includingFaceTime ™ audio. SO_TC_CTL “Network Control”, low delay tolerant, lowloss tolerant, inelastic flow, rate is bursty but short, variable size.E.g., Domain Name Service (DNS) queries; certain types oflocally-originated Internet Control Message Protocol (ICMP), ICMPv6;Internet Group Management Protocol (IGMP)/ Multicast Listener Discovery(MLD) join/leave, Address Resolution Protocol (ARP). TCP/UDPacknowledgement (ACK).

Thus, it can be seen from the above that, in one embodiment, the systemassigns network control packets to the highest priority classification,thereby ensuring that control packets are forwarded ahead of packetshaving all lesser classifications. This is an improvement over priorsystems in which certain control packets (e.g., such as TCPacknowledgement (ACK) packets could become stuck in a queue behind othertypes of non-control packets (thereby reducing system performance).

In one embodiment, any packets classified as background system initiated(SO_TC_BK_SYS) will be suspended in queues while a voice call isoccurring on the client device. As such, this embodiment providessignificant benefits over prior systems in which a voice call could bedegraded or dropped as the result of low priority packets (e.g.,background system-initiated packets) being transmitted. Thus, in thisembodiment, a user's photo stream or other data to be backed up to aservice (e.g., such as iCloud) will not interfere with voice calls.

One embodiment of the system can prevent traffic marked as backgroundsystem-initiated so that they do not interfere with an incoming phonecall, there by increasing the reliability of the call. When a phone callis initiated, the network layer (e.g., TCP/IP layer) will receive a flowcontrol notification to suspend all background system-initiated traffic.In response, the network layer may stop sending down any more packets tothe interface. It may also stop the application from writing any moredata down the network stack. This will help to improve CPU utilizationbecause the application is quiesced and it also improves reliability ofvoice calls. If the voice call completes in a reasonable duration oftime, the applications can resume data communication.

In one embodiment, when a particular lower priority application issuspended, the network stack 102 will (periodically) probe thecommunication link (e.g., via feedback signal 171) to determine when itmay resume transmission, and will communicate this information to therespective application. When the link is no longer loaded, packettransmission will then resume.

In summary, the continuous flow control feedback 171 between the driverlayer 150 and the networking stack 102, and the feedback 172 between thenetworking stack and the applications 105-107 provides for a moreintelligent, efficient allocation of network channels and bandwidth.

In one embodiment, the above values do not imply guarantees, but arerather hints from the application to the system about thecharacteristics of its traffic. The system will do its best in providingsome form of differentiations on the traffics based on theirclassifications, but no guarantee is made due to the varying factorsranging from packet scheduling parameters to network topology or mediaconditions.

In one embodiment, traffic generated by a socket associated with one ofthe above values will carry the corresponding service class value in thequeue/buffer (mbuf); there is a 1:1 mapping between SO_TC and MBUF_SCvalues, as illustrated in FIG. 8.

FIG. 9A illustrates a worst-case scenario of link-share distribution inone embodiment with the built-in QFQ scheduler configuration. Inaddition to the socket-level classification, some places throughout thekernel use the MBUF_SC_CTL service class to classify certaincontrol-type packets, (e.g., ARP, ND NS/NA, IGMP/MLD). This alsoincludes temporary elevation of service classes used by TCP ACKs.

Implicit Classification:

This form of classification is possible via the packet filter (PF)framework. It allows for the classification rules to be installed viaPF, and take effect for all IP traffics regardless of how they wereoriginally classified at the origin. PF and pfctl(8) have been enhancedwith service class-related keywords; FIG. 9B illustrates an example of aPF configuration file that can be processed by pfctl(8) to override thebuilt-in settings.

Thus, in the explicit classification case, an application opens a socketwith the default service class (BE). The application may set the serviceclass of the socket via the SO_TRAFFIC_CLASS socket option, so that allfuture send/write operations will automatically cause packets to bemarked with the corresponding service class. The application may chooseto selectively associate each datagram sent down the socket with aSO_TRAFFIC_CLASS ancillary message option, so that the associated packetwill be marked with the corresponding service class (but will not affectother current or future packets.) In this case, we can easily have manydifferent service classes associated with this socket.

In the implicit classification case, classification rules are installedin the packet filter engine. Each rule contains a signature (e.g.protocol, ports, etc) which upon a match, would result in the packet tobe marked with a service class.

Classification Tags.

In addition to marking the queue/buffer (mbuf) with a MBUF_SC value, inone embodiment, the module performing packet classification 202 alsoassociates one or more tags with the packet, in order to assist the restof the system in identifying the type or flow of the packet. In oneembodiment, these tags reside within the built-in pf_mtag sub-structureof the mbuf, and are set regardless of how the classification isperformed (explicit or implicit). The tags employed in one embodimentare as follows:

Tags Descriptions PF_TAG_FLOWHASH A flow hash has been computed andassociated with the mbuf. PF_TAG_FLOWADV The packet belongs to alocally- terminated flow which is flow advisory capable. PF_TAG_TCP Theoutermost packet uses TCP as transport.

Flow Feedback

Flow Hash:

As illustrated in FIG. 10A, in one embodiment, each mbuf that is beingsent out through an interface is tagged with a flow hash 1001 (hencemarked with PF_TAG_FLOWHASH) which will help to identify all the packetsthat belong to a particular flow at the interface layer. In oneembodiment, a flow hash 1001 is a 32-bit integer, and it is calculatedin one of the following places:

1. Socket 204. In one embodiment, when a socket is connected, the flowhash for the socket 204 is computed and stored. Further transmissions onthis socket will cause the hash value to be carried within the mbufstructure of the packets.

2. Packet Filter (PF). In one embodiment, when a packet enters thedriver 150, a flow hash will be computed and stored in the associated PFrule and states, unless it is already classified. If the packet ispassed back to IP 206 successfully, it will carry with it the flow hashassociated with the rule or state which it was matched against, in thembuf structure.

In one embodiment, the hashing algorithms used to compute the flow hashdiffers across computing system platforms depending on performance. Thefollowing table illustrates exemplary platforms and correspondinghashes:

Platforms Hash Algorithms Intel 32-bit MurmurHash3 (32-bit, i386variant) Intel 64-bit MurmurHash3 (128-bit, x86_64 variant) ARM, et al32-bit JHash

Flow Control & Advisory for TCP:

Using the queue management techniques described herein, applicationssending using TCP are flow-controlled when the number of packets perflow queued at the interface reaches an upper limit. Instead of using anindicator like explicit congestion notification (ECN) or packet drops,the interface provides a flow-advisory feedback to the transport layer.This can be done without any packet loss.

A flow advisory on a connection is received from AQM when one of thefollowing two conditions is true:

-   -   1. The send rate of a TCP connection increases above the        bandwidth supported on the link.    -   2. The available bandwidth on a wireless link, which is the        first hop, from the device goes down

In both of these cases, sending more packets will accumulate packets inthe interface queue and will increase the latency experienced by theapplication. Otherwise, it might cause packet drops which will reducethe performance because the TCP sender will have to retransmit thosepackets. By using the flow-advisory mechanism, the TCP senders can adaptto the bandwidth available without seeing any packet loss or any loss ofperformance. The interface queue will never drop a TCP packet but itwill only send a flow advisory to the connection. Because of thismechanism, buffering in device drivers was reduced by a significantamount resulting in improved latency for all TCP connections on thedevice.

The main response of a TCP connection to a flow advisory is to reduceits congestion window which will in effect reduce its sending rate. Thisis done by backing off the slow start threshold and allowing theconnection to enter congestion avoidance phase. But if the connection isalready in recovery, it means that the connection has alreadyexperienced packet loss in that round-trip time and has already loweredits sending rate. In this case, the congestion window is not reduced anyfurther.

A connection that is flow-controlled will avoid making the socketwritable until flow control is lifted. This will prevent the applicationfrom writing more data that might just get buffered up in the networkstack when it can not send packets out on the interface. This will helpinteractive applications that need to send only the latest updates andwould rather throw away the older updates.

While in a flow-controlled state, if there is an indication of packetloss in TCP acknowledgements received in the form of duplicateacknowledgements or SACK information then the connection will abortflow-control and start fast recovery to retransmit the lost data rightaway. This will not increase latency any more because the rate ofpackets sent during recovery is limited. Since the interface isguaranteed to not drop any TCP packets, the connection will be able toretransmit the lost data as quickly as possible.

When a connection is in a flow-controlled state, it means that packetsare leaving the interface slower than before. In this situation, therecan be packets waiting in the interface queue ready to be sent. Usuallythese are the packets at the end of the last send window. If this waittime is more than the retransmit timeout calculated on the connection,then a timeout will fire. At this point, retransmitting already sentdata might create duplicate copies of the same packet in the interfacequeue. This might generate duplicate acknowledgements later and causethe connection to go into recovery unnecessarily.

To avoid this confusion, a flow-controlled TCP connection will avoidretransmitting packets from a retransmit timer until later. If the waitis too long, then the connection might get timed out instead of waitingforever and an error will be returned to the application.

Every time a retransmit timeout fires, the timer is backed off beforetrying again. This will help to detect a sudden increase in delay on thelink. But for flow controlled sockets, the delay might be a result ofthe flow being blocked temporarily. When the connection comes out of theflow-controlled state, the back-off is undone. This will help to firethe retransmit timer in a timely fashion from then on.

When the packets in the interface queue flow out and the queue levelsfall below a threshold, the interface will generate a flow advisory tolet all the flows that were flow-controlled to start sending data again.At this point, the TCP sockets also become writable and the applicationscan start writing data.

When flow-control is lifted, the connection will send new data that wasnever sent before. This will generate new acknowledgements and willstart the ACK timer. It will also help to detect if there was any dataloss prior to flow-control that was not already detected. If there is nonew data to be sent, the retransmit timer will fire soon and it willtrigger retransmission of any outstanding data that has not beenacknowledged.

Using Flow advisory and flow control mechanism a TCP connection will beable to adapt to the variations in link bandwidth and will be able tominimize delay induced by buffering packets at multiple levels on thehost.

Flow Control & Advisory for UDP:

In one embodiment, a UDP socket is capable of flow control only if it isconnected to the peer using a connect( ) system call. When the number ofpackets from a UDP flow in the interface queue is more than the limitfor flow control, an advisory is generated. The socket is marked asflow-controlled at that point. Unlike TCP, the interface queue will dropall UDP packets generated thereafter. The socket will not be writeablewhich means an application waiting for write event using select or pollor kevent system classes will not get the event until flow-control islifted. If an application writes data to the socket anyways, the packetwill be dropped by the socket layer and an error (ENOBUFS) will bereturned.

This is different from the previous behavior where all UDP writessucceeded only to drop the packet later by the driver. The UDP flowcontrol and advisory will give immediate feedback to the applications sothat they can reduce their send rate immediately. For instance, a videoapplication can change it's encoding to send less data over the network.

Since the packet is dropped at the socket layer on a flow-controlled UDPsocket, it saves a lot of CPU utilization compared to the previous wherethe packet was processed and sent all the way to the driver only to bedropped. Another advantage is that a flow controlled UDP flow can notoverwhelm the interface. This will reduce cause packet loss and improvelatency for the other flows on the host.

In one embodiment, the tracking of flows at the interface layer is madepossible due to the use of Stochastic Fair Blue (SFB) (see Appendix B tothis patent application) as the queuing algorithm. In one embodiment,the implementation of SFB uses a 2-level bloom filter, whereby a flow(as indicated by the flow hash value 1001) maps to exactly one bin ateach SFB level 1009. Each bin of this embodiment tracks the number ofpackets as well as the flow drop/mark probability. In one embodiment,SFB also tracks the list of flows being flow controlled. The thresholdsfor flow control and flow advisory are based upon the bin allocation(currently set to ⅓ of the queue limit). The bin probability is updatedaccordingly, but it is currently not used for rate limiting.

Flow Suspend & Resume

In one embodiment, certain sockets that are marked as “opportunistic”are suspended when a network interface is throttled. Packets generatedby such sockets will be dropped in one embodiment when they are enqueuedon the affected queues. In one embodiment, a NOTE_SUSPENDED event willbe generated on the socket in order to inform the application thattraffics on the socket are indefinitely blocked. The Application maythen decide whether or not to abort the connection. When the interfaceis no longer throttled, the affected queues will no longer blockpackets, and a NOTE_RESUMED event will be generated on the affectedsockets. Internally, the same mechanism may be 0used by flow control andadvisory is used for implementing suspend and resume.

Inbound Network Layer Scheduling Model

Opportunistic polling of one embodiment uses a network driver inputmodel as illustrated in FIG. 11B. The driver component 1111 polls thehardware 1112 for incoming packets and the IO networking interface 1110polls the driver. Each receive queue instance polls (3) the IPnetworking interface 1110 of the driver 1111 to determine if there areany packets associated with that receive queue ready to be dequeued (3)and passed up to the network stack 1108 (4) and subsequently up to therequesting application 1107 (5) (which, as illustrated at (6) polls thereceive queue instance 1109 with which it is associated for newpackets).

Thus, with this new model, inbound packets are no longer pushed up tothe networking stack by the driver/family as illustrated by operations1-6 in FIG. 11A. Instead, inbound packets reside in the driver's receivequeue 1109 until they are dequeued by the networking stack 1108. In oneembodiment, this involves turning off the client device hardware'sreceive interrupt (IRQ). One reason the described embodiment is uniqueis that the networking stack 1108 (in conjunction with the driver)alternates between polling (FIG. 11B) and the legacy model (FIG. 11A),depending upon the load factor. In one embodiment, when the load reachesa predetermined threshold (e.g., a specified level of packets built upin the driver's queues) then the system may transition to the legacymodel (FIG. 11A). When transitioning to the legacy model, the hardware'sreceive IRQ is turned on and the driver 1105 pushes the packets up fromthe IO networking interface 1104 to the appropriate receive queueinstance 1103 and ultimately to the requesting application 1101 via thenetwork stack 1102.

Kernel Programming Interface (KPI)

In one embodiment, in order to accommodate the above configuration, aset of private KPIs is employed:

ifnet_allocate_extended( )

Allocate an ifnet instance which supports the new input model, withseveral related fields:

ifnet_init_eparams fields Descriptions input_poll( ) This callback iscalled by the network stack 1108 to retrieve one or more packets fromthe driver 1111 which implements the new driver input model. input_ctl() This callback is invoked by the network stack 1108 to inform andinstruct the driver 1111 about poll vs. legacy model transition.rcvq_maxlen The maximum size of the driver receive queue.

ifnet_input_extended( )

Analogous to ifnet_input( ) except that the driver 1111 provides thenetworking stack 1108 with all of the information related to thebeginning and end of the packet chain, as well as the total packet andbyte counts. Drivers which already possesses this information areencouraged to utilize this new variant, as it allows for betterefficiency. This may be used regardless of whether the driver adopts thenew model (FIG. 11B).

In one embodiment, a driver that has registered itself as supporting thenew input model (FIG. 11B) is flagged with the IFEF_RXPOLL flag.

Data Link Interface Layer (DLIL) 208

Input Threads:

In one embodiment, input packet processing throughout the networkingstack occurs within the context of a DLIL input thread. Some interfaceshave their own dedicated input threads, while others share a common(main) input thread. In one embodiment, there are 3 variants of DLILinput threads:

1. Main

In one embodiment, the main input thread is used by the loopbackinterface, as well as other interfaces which don't get their owndedicated input threads (i.e. anything but Ethernet/PDP or those thatdon't support RXPOLL). This thread is also used for handling allprotocol registrations and packet injections. This is implemented indlil_main_input_thread_func( )

2. Legacy

In one embodiment, legacy is used by Ethernet/PDP interfaces which donot adopt the RXPOLL model, implemented in dlil_input_thread_func( ).

3. RXPOLL

In one embodiment, RXPOLL is used by any interface which adopts theRXPOLL model, implemented in dlil_rxpoll_input_thread_func( ).

Poller Thread

In one embodiment, an interface which supports the new input model(RXPOLL) uses a dedicated kernel thread, the “poller thread”(illustrated at 1202 in FIG. 12) whose job is to invoke the driver'sinput_poll( ) callback in order to retrieve one or more packets from thedriver; this occurs when polling is ON, otherwise the poller thread 1202stays dormant. This thread is analogous to the work loop threadillustrated at 1201 in FIG. 12, where they both end up callingifnet_input( ) in order to deposit the packets to the receive queue ofthe RXPOLL-capable DLIL input thread.

The packets are then sent up the networking stack for further processingfrom the context of this DLIL input thread.

Opportunistic Polling

In one embodiment, RXPOLL-capable interfaces transition betweenIFNET_MODEL_INPUT_POLL_OFF and IFNET_MODEL_INPUT_POLL_ON modes. In oneembodiment, the former is the default/initial mode; the network stackchooses this mode for the interface when it determines that the loadfactor is low. The load factor is currently determined by looking at theEWMA of the packets and bytes in the DLIL receive queue (P_avg, B_avg)and the EWMA of the DLIL input thread wakeup requests (W_avg).

Referring to the DLL input thread 1203 in FIG. 12, in one embodiment,switching to the IFNET_MODEL_INPUT_POLL_ON is done when (P_avg≧P_hiwat&& (B_avg≧B_hiwat∥W_avg W_hiwat)), where P_hiwat, B_hiwat and W_hiwatare the high-watermark values for the packets, bytes and the wakeuprequests, respectively. Conversely, switching to theIFNET_MODEL_INPUT_POLL_OFF is done when (P_avg≦P_lowat && B_avg≦B_lowat&& W_avg≦W_lowat), where P_lowat, B_lowat and W_lowat are thelow-watermark values for the variables.

In one embodiment, these low- and high-watermark values are currentlychosen arbitrarily based on certain workloads, and they should beadjusted accordingly to accommodate future workloads (and varying linkspeeds.)

In one embodiment, The bulk of the hybrid polling logic resides withindlil_rxpoll_input_thread_func( ) where the transitions between the modestake place by calling the driver's input_ctl( ) callback based upon theabove logic. Care is taken to rate limit the transitions, such that theydo not occur too often (the hold time is set to 1 sec by default.)

FIG. 12 illustrates one embodiment of the relationship between thethreads 1201-1203 when polling is turned OFF/ON.

In one embodiment, the main difference between polling OFF/ON mode liesin the context and frequency in calling ifnet_input( ) orifnet_input_extended( ).

In one embodiment, when polling is OFF, the work loop thread 1201 getsscheduled as part of the host CPU handling a receive IRQ from thehardware; this IRQ signals the host that the device has transferred oneor more packets to the host (e.g. via DMA). Regardless of the level ofIRQ coalescing done at the hardware, the frequency of having this IOKitwork loop thread scheduled is driven by the rate of inbound packets. Thecosts associated with this scheduling (context switches, etc.) are quitesignificant, especially given the fact that our system architectureroutes all IRQs to CPU0. Therefore, in one embodiment, upon detecting ahigh load factor, polling is turned ON.

When polling is ON, the work loop thread 1201 is quiesced by virtue ofturning off the receive IRQ. Packets still get transferred to the hostfrom the device, and they accumulate in the driver's 1111's receivebuffer until they are retrieved by networking stack 1108 via theinput_poll( ) callback. The poller thread 1202, which is now active,performs the equivalent functionalities of the work loop thread 1201,except that the frequency of which this thread gets scheduled is tightlycontrolled by the networking stack 1108.

In one embodiment, polling results in improved performance given theamortizing of per-packet processing costs related to receiving packetsfrom the medium. When polling is turned ON, the network stack instructsthe driver to go into polling mode. While in polling mode, the driverwould turn OFF its receive interrupts or trap handlers associated withthe notification of packets arriving from the hardware. Packets willkeep on coming to the host's memory (from the device, via DMA orequivalent), except that the CPU will not be interrupted. This reducesthe load on the CPU as each interrupt would normally trigger a series ofwork to process it; and it would have some negative effects onperformance, as it preempts whatever is running on the CPU at the time.The network stack then polls at 1 millisecond interval (by default; thisis configurable) and pulls packets from the driver during each interval.If it detects that the packet rate has dropped, polling mode is exitedand interrupt is reenabled.

In one embodiment, polling may be employed to reduce power consumption(e.g., when in “low-power” mode) or based on user activity (orinactivity). For example, if the system is in low-power mode, thisinformation is supplied to the network stack, and the network stack maythen choose to enter polling mode on all eligible network interfaces.The network stack would then be informed when the system is no longer inlow-power mode so that polling mode can be exited.

With respect to use activity, if the system is busy handling UserInterface inputs, this information is supplied to the network stack, andthe network stack may then choose to enter polling mode on all eligiblenetwork interfaces. The network stack would be informed when the systemis no longer busy handling UI inputs, so that polling mode can beexited.

Receive Queue

In one embodiment, the if_inp member of ifnet holds the receive queuefor the DLIL input thread 1203. In one embodiment, this data structurecontains the information illustrated in FIG. 13.

Unlike its transmit counterpart, the receive queue is associated with aDLIL input thread instance, rather than with an interface. As mentionedabove, certain types of interfaces share the common (main) input thread,while others get their own dedicated input threads. Also unlike transmitwhere there may be up to N transmit queues, there is currently only 1receive queue instance per input thread. This structure also holds theinformation about the actual kernel threads used for input, work loop1201, as well as the poller threads 1202. In one embodiment, all ofthese threads are configured to share the same thread affinity tag, inorder for them to be scheduled in the same processor set (for bettercache locality). The parameters needed for opportunistic polling (e.g.mode, {P,B,W}_avg, {P,B,W}_{lo,hi}wat) also reside within this structurein one embodiment.

Link Events

In one embodiment, events related to the interface are sent from thenetworking stack 102 to the attached scheduler 116 and queue managementlogic 115, and further onto all class queue instances 110-112. Thisallows for the scheduler, queue management logic 115 and its classes toadapt their parameters accordingly, if needed. The events are asfollows:

Events Descriptions CLASSQ_EV_LINK_SPEED Effective link rate haschanged, as reported by the driver (or due to changes in TBR.)CLASSQ_EV_LINK_MTU Link MTU has changed, as resulted from SIOCSIFMTU.CLASSQ_EV_LINK_UP Link state has changed (up or CLASSQ_EV_LINK_DOWNdown), as resulted from setting/ clearing IFF_UP.

As mentioned above, embodiments of the invention include support for twodifferent modes of scheduling: (1) scheduling at the network stack layerand (2) scheduling at the driver layer. The driver can choose which typeof scheduling to use. In one embodiment, if the driver is implementing802.11n, then it may choose to perform the scheduling using the fourservice classes defined by WMM (see, e.g., FIG. 7 illustrating the 10:4mapping between service classes and queue instances) while if the driveris implementing any other interface (e.g., 102.11b/g, 3G, Ethernet, etc)it may choose to have scheduling performed at the network layer (see,e.g., FIG. 6 illustrating a 1:1 mapping between service classes andqueue instances).

In one embodiment, in the driver managed model all of the queues may beset up as with the network stack managed model, but the scheduling isperformed by the driver scheduler 160. As such, the driver-basedscheduler will then request a number of packets for each dequeueoperation for a particular class based on priority (i.e., using the 4classes for WMM).

While the schedulers 116, 160 decide which queue from which to dequeue apacket, the queuing algorithm implemented by the queue management logic115 (also referred to as the “dropper” because it has the option toeither drop or queue packets) determines which queue into which a packetshould be queued prior to dequeue. In one embodiment, the scheduler 116hands the packet to the queue management logic 115 which then determineswhether the packet is dropped or enqueued.

As mentioned above, in one embodiment, the scheduler 116 used by thenetwork stack uses quick fair queuing (QFQ) while the driver scheduler160 uses traffic class queuing (TCQ) each of which are described indetail above.

Additional Details for Flow Control and SFB Queuing

In one embodiment, the same queue management logic 115 is usedregardless of the type of scheduler selected. Stochastic fair blue (SFB)may be used as the default queueing algorithm implemented by the queuemanagement logic 115 for both the driver managed scheduler 160 and thenetwork level scheduler 116. As indicated in FIG. 10A, flow hashingallows packets from different flows/sockets to be easily tracked withinboth the network and driver layer and utilized by the SFB algorithm.When a socket 204 is initially connected, the hash value 1001 iscomputed, stored in the socket data structure and is used for thelifetime of the connected socket 204. Any time data is sent down fromthe socket to the lower layers (e.g., transport 205, IP 206, sendq 306,and driver 150), the flow hash value 1001 is sent with the packet and isused to uniquely identify the socket/flow. Once the flow is identified,a counter is incremented within one or more of the SFB bins 1009(indicated by variable C in FIG. 10A) as each new packet is queued anddecremented as each packet is dequeued. Thus, the number of packetscurrently queued for each flow is known and is used to perform flowcontrol for that flow. In one embodiment, if a particular flow haspackets queued beyond a specified threshold (as indicated by counterC>=FADV THRESHOLD in FIG. 10A), then a probability value for that flowis incremented by some interval. Once the probability reaches its limit(e.g., 1), this indicates that there are too many packets (i.e., theapplication is sending too fast) and further packets for this flow aredropped.

FIG. 10A illustrates this mechanism in detail. In one embodiment, theflow hash is used to tag every packet that goes down from the socket 204to the lower layers, so these lower layers can recognize which packetsbelow to which flow and provide feedback to the network layer. Packetsare queued within the sendq 306 associated with the flow and, asmentioned, SFB uses the flow hash to increment and decrement thecounters (C₁, C₂) stored within the SFB bins 1009 for each flow. SFBuses the counters in its in memory data structures to determine if thequeuing threshold has been exceeded. For example, if the number ofpackets queued for a flow is 50, which is at the threshold, it storesstate for that flow and sends feedback to the transport layer, turningon flow control for the socket 204 which supports that flow. Thus, theflow moves from a normal to a flow controlled state and remains in thisstate while packets get dequeued from the queue and sent over the link.

Once the number of packets in the queue drops below a threshold, thequeuing algorithm wakes a flow advisor thread 1050, which wakes up andturns off flow control for the socket 204. Consequently, the socket maythen transmit packets down through the lower layers at will, until it isplaced in a flow controlled state again. Thus, FIG. 10A illustrates amechanism which provides feedback and allows flows on the same host tobe moved between a flow controlled/normal state without relying oninternal mechanisms in TCP such as explicit congestion notification(ECN). This improves performance because the unnecessary work done bysending data down to the network stack just to be dropped is reduced. Inaddition, it allows the queue size in the driver to be decreased withoutpacket drops. This is in contrast to the older mechanisms in which theapplication blindly transmits packets down the stack only to be droppedby the driver. In contrast, in FIG. 10A, since the network stack knowswhat is going on the link because of queue monitoring, it can use thisinformation to intelligently change the behavior of the transportmechanism.

In one embodiment illustrated in greater detail in FIG. 10B, the SFBalgorithm takes the hash value of the packet and computes another hashwhich is unique for SFB and which is used to pick the bits for the flow.As illustrated, the original hash value computed by the socket is 4bytes (01, 02, 03, 04) this may be provided to the hash generator 1065of the SFB queuing logic 1060. Certain flow hashes may be provided witha value of 0 and used to identify flows for which the flow controlmechanisms described in here are not used. In one embodiment, SFBgenerates a random number and computes the new hash using the randomnumber (A, B, C, D). It uses these four values to populate two sets ofbins: a current set and a shadow set, each of which has two levels, asshown in FIG. 10B. The two levels are implemented as an array which isindexed using the new hash values (each slot is 256 bits in oneembodiment). Thus, the flow is mapped into the SFB bins using the flowhash value. To check whether a flow exists, the SFB bins may be checked.The bloom filter functionality of the SFB algorithm is capable ofchecking whether a particular flow does not exist within the SFB bins.If one exists, it can be indexed to determine the data associated withthat flow (e.g., the probability and counters mentioned above). Thisdata is maintained by SFB as long as a flow still exists. The shadow setis used because at some interval SFB rehashes. Because of the propertyof a bloom filter, many things can map to the same bin. If there are alot of sockets, some sockets that are sending data too much can impactother sockets that are not sending a lot of data which are mapped to thesame bin. Thus, SFB periodically re-computes the internal hash based ona new random number. After a rehash, the bins may move around for eachflow. Additional details of the SFB algorithm can be found in thereference Stochastic Fair Blue: A Queue Management Algorithm forEnforcing Fairness which is attached as Appendix B to this patentapplication and incorporated herein by reference. In this case, theshadow set may be used to move the data to the new bins.

Application-Driven Traffic Classifications and Flow Feedback, IncludingOpportunistic Behavior, Flow Control, Traffic Suspension, and TransportLayer Improvements

The application-driven traffic classifications described above withrespect to FIGS. 6-8, may be used to enable various opportunisticbehaviors. As mentioned, each application may explicitly state what thetraffic class should be for its sockets. For example, say oneapplication is backing up date using a low priority traffic class suchas BK_SYS but the user wants to make a voice call. Because of the wayGSM works, if a voice call is in progress while network data is beingsent, then this increases the probability of the voice call todeteriorate or drop. Similarly, if a user opens a web browser while on acall, then it may also impact the call. To alleviate this, in oneembodiment, traffic classes are used to suppress unnecessary trafficwhen a user is on a voice call. Any non-urgent traffic (e.g., backgroundtraffic—BKS_SYS) may be suspended during the voice call. In oneembodiment, for suspended sockets, when a dequeue happens, the systempretends that there is nothing to dequeue and when an enqueue happens,packets are dropped. Thus, the transmitting application will see this asif the network has stopped and may wait before trying again. If the callis relatively quick, then it will resume when the call has ended(without shutting down the socket entirely). These techniques may beused for traffic classes other that BK_SYS (e.g., BK, BE, etc). In oneembodiment, the flow control/advisory mechanism described above is alsoused to suspend sockets in these situations (e.g., if suspended, a flowcontrol indication is sent). Thus, by classifying traffic for certainsockets (e.g., BK_SYS) an intelligent feedback mechanism is provided forhandling traffic suspension and opportunistic behavior for these socketsis enabled in the kernel.

Note that the suspended state is different from the flow controlledstate. In the flow controlled state, the application may still besending data but in the suspended state, the link is blocked (e.g.,because a voice call takes a long time). Thus, in the suspended state,it is beneficial to stop sending any more packets because they will justget dropped (because the queue is suspended). In one embodiment, thesuspended application may choose to start a timer and, if suspended toolong, simply close the connection.

In summary, in one embodiment, when a voice call is received/made:

(1) An authoritative entity in the system configures the cellularnetwork interface(s) into opportunistic throttling mode.

(2) The opportunistic mode gets configured on the packet scheduler ofeach affected interface.

(3) The scheduler goes through all transmit queues that are to besuspended in this opportunistic mode; at present this applies only toBK_SYS transmit queue. Each affected transmit queue goes into asuspended mode; all existing packets are flushed, and further enqueueswill cause drops.

(4) All opportunistic sockets (sockets with BK_SYS classification) willreceive a “suspend event.”

In one embodiment, when a voice call is terminated:

(1) An authoritative entity in the system removes opportunisticthrottling mode from the cellular network interface(s).

(2) The packet scheduler for each affected interface exits opportunisticthrottling mode.

(3) The scheduler goes thru all transmit queues that were suspended inthis opportunistic mode; at present this applies only to BK_SYS transmitqueue. Each affected transmit queue is resumed; further enqueues areallowed.

(4) All opportunistic sockets (sockets with BK_SYS classification) willreceive a “resume event.”

In addition, performance optimizations are realized for the receiveside, referred to herein as “large receive offload.” In one embodiment,this works by reducing the per packet cost by calling functions in thenetwork stack. For example, 10 packets are received, rather than dealingwith all 10, only 1 or 2 may be processed. For certain classes such asthe AV class (streaming video), we can enable this optimization. With AVvideo applications the software buffers video for several seconds beforeit starts to play. So because of this, this application may receiveperformance benefits using the large receive overload techniques,because they not delay sensitive.

In one embodiment, because of the feedback provided up the stack, themanner in which TCP sends and receives may be adjusted for certainsockets. For example, if the application classifies a flow as BK orBK_SYS, TCP will be less aggressive. For example, if a socket is BK andhigh congestion on this link is detected, the application may back offand receive a better response time for other flows (e.g., a socketbacking up to the network may be delayed to receive HTTP on anothersocket). All of this is possible because applications are able toexplicitly specify socket traffic classes.

As previously mentioned, in one embodiment, the highest classification(CTL) is used for network control packets which may include ARP, ACKresponses, neighbor discovery for IPV6, multicast join and leave,DCP-related packets, IPV6 station router advertisement packets, and DNSpackets. Consequently, DNS operations will not be delayed if the user isperforming a large upload using a lower priority socket. Similarly, ACKresponses will not be delayed as the result of activity on lowerpriority sockets. Control packets of this type are very small and can goout without any delay.

In addition, as mentioned, the techniques and architecture describedabove provide for built in TCP and UDP backoff mechanisms during flowcontrol and suspension states. Usually TCP responds to congestion itselfby backing off (e.g., using ECN as described above). TCP sends a packetand if does not get an acknowledgement (ACK), it sends the packet againand again and backs off with each retransmission. This retransmission isa waste of resources if it is known that the packet is still in theinterface queue 151-153 and will be sent only after the link becomesavailable. Thus, in this situation, there is no need to back off andretransmit using the built in TCP or UDP mechanisms. Instead, using thetechniques described herein, a flow control advisory may be issued todisable the retransmit function.

In addition, in one embodiment, the retransmit timer used for TCP may betweaked to operate more efficiently. Typically, TCP backs off theretransmit timer when no ACKs are received (e.g., increasing by 2× eachtime). Thus, retransmit timer can become several seconds. Since feedbackfrom the interface is provided as described above, this backoff does notneed to occur. Once we get a flow control advisory, we can automaticallywait because we know the packet is queued (and don't want to keepresending). However, in one embodiment, the retransmit timer may be usedafter the flow control has been turned off for the socket.

In addition, the flow advisory is propagated all the way to theapplication which does not write any more data to TCP. The applicationcould therefore drop all of the stale data (e.g., old video frames) andsend newer data when flow control is turned off. This is particularlybeneficial for interactive video data, where audio and video may be outof sync if the application does not drop stale data.

Similar principles may be applied to applications which transmit withUDP. Without the flow advisory techniques described herein, UDP does notprovide any feedback to its applications. Thus, if the interface drops,the application does not know and continues to transmit packets, whichmust therefore be buffered at the lower levels of the stack, wastingresources. By contrast, when in a flow controlled state, all of thewrites by the application may be dropped right away (thereby savingbuffering resources). Doing the above reduces a significant amount ofdelay which would result by buffering packets.

Opportunistic Polling Mechanisms for Handling Inbound Network Traffic,Improving Network Performance and Handling Denial-of-Service Attacks

As mentioned above, in one embodiment, RXPOLL-capable interfacestransition between an input polling off mode and an input polling on (orlegacy) mode of operation (e.g., IFNET_MODEL_INPUT_POLL_OFF andIFNET_MODEL_INPUT_POLL_ON. In one embodiment, the input polling off modeis used as a default/initial mode and is selected by the network stackwhen it determines that the load factor is low. The load factor may bedetermined by evaluating the variables P_avg, B_avg, and W_avg (thevalues for the number of packets, bytes and the wakeup requests,respectively). Referring to the DLL input thread 1203 in FIG. 12, in oneembodiment, switching to the input poling on mode is done when(P_avg≧P_hiwat && (B_avg≧B_hiwat∥W_avg≧W_hiwat)), where P_hiwat, B_hiwatand W_hiwat are where P_hiwat, B_hiwat and W_hiwat are low-watermarkvalues for the variables.

Conversely, switching to the polling off mode is done when(P_avg≦P_lowat && B_avg≦B_lowat && W_avg≦W_lowat), where P_lowat,B_lowat and W_lowat are the low-watermark values for the variables.

In one embodiment, these low- and high-watermark values may be chosenarbitrarily based on certain workloads, and they should be adjustedaccordingly to accommodate future workloads (and varying link speeds.)

Turning polling on in this manner, based on a high load factor, improvesperformance and prevents denial of service attacks. This is because,using the polling mechanism, the receive queue at the network layer willonly request packets from the queues at the driver layer when it hassufficient room to queue them. When it does not have room, it will notpoll to request more packets and packets to be dropped by the interface(i.e., when the driver layer queues are filled up). Thus, denial ofservice is prevented at the driver layer and is not propagated up to thenetwork layers.

Different API Embodiments

The API implemented in one embodiment, is an interface implemented by asoftware component (hereinafter “API implementing software component”)that allows a different software component (hereinafter “API callingsoftware component”) to access and use one or more functions, methods,procedures, data structures, and/or other services provided by the APIimplementing software component. For example, an API allows a developerof an API calling software component (which may be a third partydeveloper) to leverage specified features provided by an APIimplementing software component. There may be one API calling softwarecomponent or there may be more than one such software component. An APIcan be a source code interface that a computer system or program libraryprovides in order to support requests for services from a softwareapplication. An API can be specified in terms of a programming languagethat can be interpretative or compiled when an application is built,rather than an explicit low level description of how data is laid out inmemory.

The API defines the language and parameters that API calling softwarecomponents use when accessing and using specified features of the APIimplementing software component. For example, an API calling softwarecomponent accesses the specified features of the API implementingsoftware component through one or more API calls (sometimes referred toas function or method calls) exposed by the API. The API implementingsoftware component may return a value through the API in response to anAPI call from an API calling software component. While the API definesthe syntax and result of an API call (e.g., how to invoke the API calland what the API call does), the API typically does not reveal how theAPI call accomplishes the function specified by the API call. Variousfunction calls or messages are transferred via the one or moreapplication programming interfaces between the calling software (APIcalling software component) and an API implementing software component.Transferring the function calls or messages may include issuing,initiating, invoking, calling, receiving, returning, or responding tothe function calls or messages. Hence, an API calling software componentcan transfer a call and an API implementing software component cantransfer a call.

By way of example, the API implementing software component 2010 and theAPI calling software component may be an operating system, a library, adevice driver, an API, an application program, or other software module(it should be understood that the API implementing software componentand the API calling software component may be the same or different typeof software module from each other). The API calling software componentmay be a local software component (i.e., on the same data processingsystem as the API implementing software component) or a remote softwarecomponent (i.e., on a different data processing system as the APIimplementing software component) that communicates with the APIimplementing software component through the API over a network. Itshould be understood that an API implementing software component mayalso act as an API calling software component (i.e., it may make APIcalls to an API exposed by a different API implementing softwarecomponent) and an API calling software component may also act as an APIimplementing software component by implementing an API that is exposedto a different API calling software component.

The API may allow multiple API calling software components written indifferent programming languages to communicate with the API implementingsoftware component (thus the API may include features for translatingcalls and returns between the API implementing software component andthe API calling software component); however the API may be implementedin terms of a specific programming language.

FIG. 14 illustrates one embodiment of an API architecture which includesan API implementing software component 1410 (e.g., an operating system,a library, a device driver, an API, an application program, or othersoftware module) that implements the API 1420. The API 1420 specifiesone or more functions, methods, classes, objects, protocols, datastructures, formats and/or other features of the API implementingsoftware component that may be used by the API calling softwarecomponent 1430. The API 1420 can specify at least one calling conventionthat specifies how a function in the API implementing software componentreceives parameters from the API calling software component and how thefunction returns a result to the API calling software component. The APIcalling software component 1430 (e.g., an operating system, a library, adevice driver, an API, an application program, or other softwaremodule), makes API calls through the API 1420 to access and use thefeatures of the API implementing software component 1410 that arespecified by the API 1420. The API implementing software component 1410may return a value through the API 1420 to the API calling softwarecomponent 1430 in response to an API call.

It will be appreciated that the API implementing software component 1410may include additional functions, methods, classes, data structures,and/or other features that are not specified through the API 1420 andare not available to the API calling software component 1430. It shouldbe understood that the API calling software component 1430 may be on thesame system as the API implementing software component 1410 or may belocated remotely and accesses the API implementing software component1410 using the API 1420 over a network. While FIG. 14 illustrates asingle API calling software component 1430 interacting with the API1420, it should be understood that other API calling softwarecomponents, which may be written in different languages (or the samelanguage) than the API calling software component 1430, may use the API1420.

The API implementing software component 1410, the API 1420, and the APIcalling software component 1430 may be stored in a machine-readablemedium, which includes any mechanism for storing information in a formreadable by a machine (e.g., a computer or other data processingsystem). For example, a machine-readable medium includes magnetic disks,optical disks, random access memory; read only memory, flash memorydevices, etc.

In FIG. 15 (“Software Stack”), an exemplary embodiment, applications canmake calls to Services 1 or 2 using several Service APIs and toOperating System (OS) using several OS APIs. Services 1 and 2 can makecalls to OS using several OS APIs.

Note that the Service 2 has two APIs, one of which (Service 2 API 1)receives calls from and returns values to Application 1 and the other(Service 2 API 2) receives calls from and returns values to Application2. Service 1 (which can be, for example, a software library) makes callsto and receives returned values from OS API 1, and Service 2 (which canbe, for example, a software library) makes calls to and receivesreturned values from both OS API 1 and OS API 2. Application 2 makescalls to and receives returned values from OS API 2.

Exemplary Data Processing Devices and Interfaces

FIG. 16 is a block diagram illustrating an exemplary computer systemwhich may be used in some embodiments of the invention. It should beunderstood that while FIG. 16 illustrates various components of acomputer system, it is not intended to represent any particulararchitecture or manner of interconnecting the components as such detailsare not germane to the present invention. It will be appreciated thatother computer systems that have fewer components or more components mayalso be used with the present invention.

As illustrated in FIG. 16, the computer system 2300, which is a form ofa data processing system, includes the bus(es) 2350 which is coupledwith the processing system 2320, power supply 2325, memory 2330, and thenonvolatile memory 2340 (e.g., a hard drive, flash memory, Phase-ChangeMemory (PCM), etc.). The bus(es) 2350 may be connected to each otherthrough various bridges, controllers, and/or adapters as is well knownin the art. The processing system 2320 may retrieve instruction(s) fromthe memory 2330 and/or the nonvolatile memory 2340, and execute theinstructions to perform operations as described above. The bus 2350interconnects the above components together and also interconnects thosecomponents to the optional dock 2360, the display controller & displaydevice 2370, Input/Output devices 2380 (e.g., NIC (Network InterfaceCard), a cursor control (e.g., mouse, touchscreen, touchpad, etc.), akeyboard, etc.), and the optional wireless transceiver(s) 2390 (e.g.,Bluetooth, WiFi, Infrared, etc.).

FIG. 17 is a block diagram illustrating an exemplary data processingsystem which may be used in some embodiments of the invention. Forexample, the data processing system 2400 may be a handheld computer, apersonal digital assistant (PDA), a mobile telephone, a portable gamingsystem, a portable media player, a tablet or a handheld computing devicewhich may include a mobile telephone, a media player, and/or a gamingsystem. As another example, the data processing system 2400 may be anetwork computer or an embedded processing device within another device.

According to one embodiment of the invention, the exemplary architectureof the data processing system 2400 may used for the mobile devicesdescribed above. The data processing system 2400 includes the processingsystem 2420, which may include one or more microprocessors and/or asystem on an integrated circuit. The processing system 2420 is coupledwith a memory 2410, a power supply 2425 (which includes one or morebatteries) an audio input/output 2440, a display controller and displaydevice 2460, optional input/output 2450, input device(s) 2470, andwireless transceiver(s) 2430. It will be appreciated that additionalcomponents, not shown in FIG. 24, may also be a part of the dataprocessing system 2400 in certain embodiments of the invention, and incertain embodiments of the invention fewer components than shown in FIG.16 may be used. In addition, it will be appreciated that one or morebuses, not shown in FIG. 16, may be used to interconnect the variouscomponents as is well known in the art.

The memory 2410 may store data and/or programs for execution by the dataprocessing system 2400. The audio input/output 2440 may include amicrophone and/or a speaker to, for example, play music and/or providetelephony functionality through the speaker and microphone. The displaycontroller and display device 2460 may include a graphical userinterface (GUI). The wireless (e.g., RF) transceivers 2430 (e.g., a WiFitransceiver, an infrared transceiver, a Bluetooth transceiver, awireless cellular telephony transceiver, etc.) may be used tocommunicate with other data processing systems. The one or more inputdevices 2470 allow a user to provide input to the system. These inputdevices may be a keypad, keyboard, touch panel, multi touch panel, etc.The optional other input/output 2450 may be a connector for a dock.

Embodiments of the invention may include various steps as set forthabove. The steps may be embodied in machine-executable instructionswhich cause a general-purpose or special-purpose processor to performcertain steps. Alternatively, these steps may be performed by specifichardware components that contain hardwired logic for performing thesteps, or by any combination of programmed computer components andcustom hardware components.

Elements of the present invention may also be provided as amachine-readable medium for storing the machine-executable program code.The machine-readable medium may include, but is not limited to, floppydiskettes, optical disks, CD-ROMs, and magneto-optical disks, ROMs,RAMs, EPROMs, EEPROMs, magnetic or optical cards, or other type ofmedia/machine-readable medium suitable for storing electronic programcode.

Throughout the foregoing description, for the purposes of explanation,numerous specific details were set forth in order to provide a thoroughunderstanding of the invention. It will be apparent, however, to oneskilled in the art that the invention may be practiced without some ofthese specific details. For example, it will be readily apparent tothose of skill in the art that the functional modules and methodsdescribed herein may be implemented as software, hardware or anycombination thereof. Moreover, although embodiments of the invention aredescribed herein within the context of a mobile computing environment(i.e., using mobile devices 120-123; 601-603), the underlying principlesof the invention are not limited to a mobile computing implementation.Virtually any type of client or peer data processing devices may be usedin some embodiments including, for example, desktop or workstationcomputers. Accordingly, the scope and spirit of the invention should bejudged in terms of the claims which follow.

What is claimed is:
 1. A client device having a networking layer and anetwork driver layer for transmitting network packets comprising: aplurality of transmit queues configured at the network layer, each ofthe transmit queues having different packet service classificationsassociated therewith; a classifier module for classifying packetsaccording to the different packet service classifications, wherein apacket to be transmitted is stored in one of the transmit queues basedon the packet service classifications, wherein one or more of thetraffic service classifications comprise opportunistic traffic classesand other traffic service classifications comprise non-opportunistictraffic classes; and a socket layer comprising socket transmitting datafrom an application associated with each traffic service classification,wherein a socket associated with an opportunistic traffic class isautomatically suspended in response to detecting a network interface ofthe client device being throttled.
 2. The client device as in claim 1wherein the non-opportunistic traffic classes include a traffic classfor voice or video telephone calls over a cellular network and whereinthe opportunistic traffic class is a background system-initiated trafficclass.
 3. The client device as in claim 2 wherein the backgroundsystem-initiated traffic class is used for transmitting data backupsfrom the client device to a server over a network or other non-urgentdata traffic.
 4. The client device as in claim 1 wherein N packetservice classes are mapped to M transmit queues, wherein N is greaterthan or equal to M.
 5. The client device as in claim 1 wherein packetservice classifications are explicitly set by each application.
 6. Theclient device as in claim 1 wherein each packet service classificationhas a different relative priority associated therewith.
 7. The clientdevice as in claim 6 wherein network control packets are assigned to apacket service classification having the highest relative prioritylevel.
 8. The client device as in claim 1 further comprising a networkdriver layer in communication with the networking layer, wherein when anetwork packet is ready to be transmitted from a particular queue, thenetworking layer notifies the driver layer which then dequeues thepacket from the queue and transmits the packet over the network.
 9. Theclient device as in claim 1 wherein packet service classifications arebased on one or more of a delay tolerance level specifying how tolerantan application is to packet delay, a loss tolerance level specifying howtolerant an application is to packet loss, and an elasticity of flowlevel specifying how tolerant an application is to changes in bandwidth.10. The client device as in claim 9 wherein packet serviceclassifications having relatively lower delay tolerance level associatedtherewith are given relatively higher priority compared with packetservice classifications having relatively higher delay tolerance levels.11. The client device as in claim 8 wherein packet serviceclassifications having relatively lower loss tolerance associatedtherewith are given relatively higher priority compared with packetservice classifications having relatively higher loss tolerance.
 12. Theclient device as in claim 9 wherein packet service classificationshaving relatively lower elasticity of bandwidth associated therewith aregiven relatively higher priority compared with packet serviceclassifications having relatively higher elasticity of bandwidth. 13.The client device as in claim 9 wherein one or more of the followingpacket service classifications are used: background system-initiated;background; best effort; responsive data; operations, administration,and management; multimedia audio/video streaming; responsive multimediaaudio/video; interactive video; interactive audio; and network control.14. The client device as in claim 13 wherein an event indicating asuspension of an opportunistic socket is generated on the socket inorder to inform the application using the socket that traffic on thesocket is indefinitely blocked.
 15. The client device as in claim 14wherein the application makes a determination as to whether to abort thesocket connection or wait for the suspension to be removed from thesocket.
 16. The client device as in claim 1 wherein, when the socket issuspended, any new packets generated on the socket will be droppedbefore being queued and any packets previously queued will be flushed.17. The client device as in claim 16 wherein when the network interfaceis no longer throttled, the suspension is removed and packets will bequeued rather than dropped.
 18. The client device as in claim 1 furthercomprising program code to cause the processor to perform the operationsof: receiving a packet to be transmitted from an application executed onthe client device; calculating a flow hash value for the packet, theflow hash value associating the packet with a particular network flowassociated with an opportunistic socket, the flow hash value usable totrack the packet and other packets associated with the particularnetwork flow; receiving feedback for the flow from a driver layer of theclient device, the flow being identified using the flow hash value, thefeedback indicating the throttling of the network interface; and settingthe socket to a suspended state responsive to the feedback for the flow.19. The client device as in 18 wherein calculating the flow hash valuecomprises calculating a 32-bit MurmurHash3 value on the packet.
 20. Theclient device as in 18 wherein calculating the flow hash value comprisescalculating a 128-bit MurmurHash3 value on the packet.
 21. The clientdevice as in claim 18 wherein calculating the flow hash value comprisescalculating a 32-bit JHash value on the packet.
 22. The client device asin claim 18 wherein the feedback indicates a number of packets queuedfor the flow and a service class associated with the queue and whereinwhen the number of packets queued reaches a specified threshold, thendelaying the sending of additional data packets until the number ofpackets queued is reduced below the specified threshold.
 23. The clientdevice as in claim 22 wherein a socket associated with the flow ismarked as flow controlled at the network layer, the flow control markindicating that new packets for the socket should be temporarilydelayed.
 24. The client device as in claim 22 wherein if an applicationassociated with the flow continues to send data, packets for the floware dropped if the socket is a non-TCP socket.
 25. The client device asin claim 18 wherein the flow control operations comprise tagging asocket associated with the flow as flow controlled and notifying theapplication associated with the flow of the flow controlled state. 26.The client device as in claim 25 wherein the application modifies therate at which it transmits data to the socket when in a flow controlledstate.
 27. The client device as in claim 26 wherein the applicationdelays packet transmission until the flow controlled tag has beenremoved from the socket.
 28. The client device as in claim 27 wherein,when the application resumes transmitting data to the socket, it dropsone or more packets which have become stale while the application waswaiting.
 29. The client device as in claim 28 wherein the application isa real time video application and wherein the stale packets are videopackets which contain old video data.
 30. A method for transmittingnetwork packets comprising: configuring a plurality of transmit queuesat a network layer, each of the transmit queues having different packetservice classifications associated therewith; classifying packetsaccording to the different packet service classifications, wherein apacket to be transmitted is stored in one of the transmit queues basedon the packet service classifications, wherein one or more of thetraffic service classifications comprise opportunistic traffic classesand other traffic service classifications comprise non-opportunistictraffic classes; and transmitting data from socket associated with anapplication associated with each traffic service classification, whereina socket associated with an opportunistic traffic class is automaticallysuspended in response to detecting a network interface of a clientdevice being throttled.
 31. The method as in claim 30 wherein thenon-opportunistic traffic classes include a traffic class for voice orvideo telephone calls over a cellular network and wherein theopportunistic traffic class is a background system-initiated trafficclass.
 32. The method as in claim 31 wherein the backgroundsystem-initiated traffic class is used for transmitting data backupsfrom the client device to a server over a network or other non-urgentdata traffic.
 33. The method as in claim 30 wherein N packet serviceclasses are mapped to M transmit queues, wherein N is greater than orequal to M.
 34. The method as in claim 30 wherein packet serviceclassifications are explicitly set by each application.
 35. The methodas in claim 30 wherein each packet service classification has adifferent relative priority associated therewith.
 36. The method as inclaim 35 wherein network control packets are assigned to a packetservice classification having the highest relative priority level. 37.The method as in claim 30 further comprising a network driver layer incommunication with the networking layer, wherein when a network packetis ready to be transmitted from a particular queue, the networking layernotifies the driver layer which then dequeues the packet from the queueand transmits the packet over the network.
 38. The method as in claim 30wherein packet service classifications are based on one or more of adelay tolerance level specifying how tolerant an application is topacket delay, a loss tolerance level specifying how tolerant anapplication is to packet loss, and an elasticity of flow levelspecifying how tolerant an application is to changes in bandwidth. 39.The method as in claim 38 wherein packet service classifications havingrelatively lower delay tolerance level associated therewith are givenrelatively higher priority compared with packet service classificationshaving relatively higher delay tolerance levels.
 40. The method as inclaim 37 wherein packet service classifications having relatively lowerloss tolerance associated therewith are given relatively higher prioritycompared with packet service classifications having relatively higherloss tolerance.
 41. The method as in claim 38 wherein packet serviceclassifications having relatively lower elasticity of bandwidthassociated therewith are given relatively higher priority compared withpacket service classifications having relatively higher elasticity ofbandwidth.
 42. The method as in claim 38 wherein one or more of thefollowing packet service classifications are used: backgroundsystem-initiated; background; best effort; responsive data; operations,administration, and management; multimedia audio/video streaming;responsive multimedia audio/video; interactive video; interactive audio;and network control.
 43. The method as in claim 42 wherein an eventindicating a suspension of an opportunistic socket is generated on thesocket in order to inform the application using the socket that trafficon the socket is indefinitely blocked.
 44. The method as in claim 43wherein the application makes a determination as to whether to abort thesocket connection or wait for the suspension to be removed from thesocket.
 45. The method as in claim 30 wherein, when the socket issuspended, any new packets generated on the socket will be dropped andany packets previously queued will be flushed.
 46. The method as inclaim 35 wherein when the network interface is no longer throttled, thesuspension is removed and packets will be queued rather than dropped.47. The method as in claim 30 further comprising program code to causethe processor to perform the operations of: receiving a packet to betransmitted from an application executed on the client device;calculating a flow hash value for the packet, the flow hash valueassociating the packet with a particular network flow associated with anopportunistic socket, the flow hash value usable to track the packet andother packets associated with the particular network flow; receivingfeedback for the flow from a driver layer of the client device, the flowbeing identified using the flow hash value, the feedback indicating thethrottling of the network interface; and setting the socket to asuspended state responsive to the feedback for the flow.
 48. The methodas in 47 wherein calculating the flow hash value comprises calculating a32-bit MurmurHash3 value on the packet.
 49. The method as in 47 whereincalculating the flow hash value comprises calculating a 128-bitMurmurHash3 value on the packet.
 50. The method as in claim 37 whereincalculating the flow hash value comprises calculating a 32-bit JHashvalue on the packet.
 51. A machine-readable medium having program codestored thereon which, when executed by a processor, causes the processorto perform the operations of: opening an opportunistic network socket ona data processing device; detecting an active voice call over a cellularnetwork interface on a data processing device; configuring the cellularnetwork interface(s) into an opportunistic throttling mode; wherein eachaffected transmit queue responsively enters into a suspended modewherein existing packets in the transmit queue are flushed, and furtherenqueue attempts will cause packet drops; and wherein all opportunisticsockets receive a suspend event to provide a notification that anyfurther enqueue attempts will cause packet drops.
 52. Themachine-readable medium as in claim 51 wherein the opportunistic mode isconfigured on a packet scheduler of each affected interface such thatthe packet scheduler identifies transmit queues affected by theopportunistic mode.
 53. The machine readable medium as in claim 51comprising additional program code to cause the processor to perform theoperations of: detecting that the voice call is terminated; removingopportunistic throttling mode from the cellular network interface(s);wherein each affected transmit queue responsively exit from thesuspended mode such that the queues may receive new packets; and whereinthe opportunistic sockets receive a resume event indicating that datatransmission may resume.
 54. The machine-readable medium as in claim 51wherein one type of opportunistic socket comprises a socket having aclassification of background system-initiated (BK_SYS).